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An  algorithm  Is  present'd  for  determining  the  values  which 
slaultanaouslg  satisfy  a set  of  relations,  or  constraints,  involving 
subsets  of  n variables.  The  relations  are  repreeented  in  a 
eeriee  of  constraint  networks,  which  ultimately  contain  a node  for  everu 
eubset  of  the  n variables.  Constraints  aay  be  propagated  through  euch 
networks  In  (potentially)  parallel  fashion  to  determine  the  valuee  which 
simul taneouely  satisfy  all  the  constraints.  The  iterated  conetraint 
propagation  serves  to  mitigate  combinatorial  explosion.  Applications  in 
scene  analysis,  graph  theory,  and  backtrack  search  are  provided. 
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1.  Satisfying  simultaneous  cons train tat  problea  and  applications 

Us  ars  givsn  a sat  of  varlablss  Xj,...,Xn  and  constraints  on  subssts 
of  thsss  varlablss  Halting  tha  values  they  can  take  on.  These  constrainte 
taken  together  constitute  a global  constraint  uhich  specifies  uhich  sets  of 
values  a^ , . . . , an  for  Xj,,..,Xn  can  simultaneously  satisfy  all  the  given 
constraints.  In  other  uords,  the  constraints  define  an  n-ary  relation. 
Our  problea  is  to  synthesize  this  relation,  i.s.  to  determine  those  sets  of 
values  uhich  simultaneously  satisfy  the  set  of  constraints. 

The  simultaneous  satisfaction  of  several  constraints — call  them 
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properties,  relationships,  predicates,  features  or  attributes— is  a very 
general  problem,  uith  more  applications  than  I can  fully  survey  here.  The 
essential  technique  ue  apply,  iterated  reduction  of  possibilities  through 
constraint  propagation,  has  analogues  in  many  areas  of  computer  science  and 
mathematics.  Many  of  these  applications  and  analogues  are  described  in 
US),  (111  and  124}.  Applications  range  from  data  base  retrieval  (find  all 
*.  y and  z such  that  k Is  a part  and  y is  a part,  and  z is  a supplier,  x 
must  be  installed  before  y,  and  z supplies  both  x and  y)  (see  also  (121)  to 
scene  analysis  (segment  the  scene  into  regions  such  that  sky  regions  are 
blue,  grass  regions  ars  green,  and  car  regions  ars  shiny,  sky  regions  are 
above  graes  regions  and  cars  ars  not  totally  surrounded  by  either  grass  or 
aky).  Of  particular  note  is  the  uork  of  J.R.  Ullman,  uho  has  used 
conetraint  propagation  methods  in  a variety  of  contexts,  ranging  from 
pattarn  recognition  (19)  to  graph  Isomorphism  (21).  The  problem  also 
admits  of  a graphical  representation,  uhere  its  resemblance  to  networks  of 
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interacting  procaasaa  conjures  up  a long  history  of  other  work,  including 
recently!  (81,  (18)  and  (6). 

In  scene  analysis,  in  particular,  thare  has  been  a recent  groundsuell 
of  appl  1 cat  ions,  e.g.  til,  [161,  (91,  (131.  Several  of  the  latest  examples 
can  be  found  in  [31. 

Often  ue  are  only  given,  or  choose  to  use,  "local"  constraints,  i.e. 
constraints  on  small  subsets  of  the  variables,  from  uhleh  ue  must 
eynthesize  the  global  constraint.  For  fundamental  reeults  on  the 
complementary  problem,  analysis  of  a global  constraint  into  local  ones,  see 
(141 . 


2.  Previous  results!  partial  consistency 

Constraints  represented  in  network  form  may  be  propagated  through 
(potentially)  parallel  algorithms  which  cut  down  the  solution  search  space 
by  ruling  out  Inconsistent  combinations  of  values. 

The  obvious  brute  force  approach  of  testing  svsry  possible  combination 
of  values  faces  an  equally  obvious  combinatorial  explosion.  Backtrack 
search  techniques  cut  down  the  search  space  but  often  exhibit  costly 
"thrashing"  behavior  1171  (2).  tlackuorth  (111  has  interpreted  previous 
work  by  Fikes  (7J,  Waltz  (23)  and  flontanari  (1S1  as  cutting  down  the  search 
space  and  avoiding  classes  of  thrashing  behavior  by  eliminating 
combinations  of  values  which  could  not  eppear  together  In  any  eet 
satisfying  the  global  constraint. 


A network  representat Ion  of  a set  of  conatrainta  ia  employed 


(raatrictad  to  unary  and  binary  conatrainta,  pradlcataa  on  ona  or  tuo 
variablaa).  Each  variable  la  ropreaented  by  a node,  and  each  binary 
predicate  by  a link  or  arc  betueen  tuo  nodea.  (Loops  on  a node  aiay  be 
vieued  ae  binary  or  unary  predicates.)  For  exaeple,  the  probleia  of 
coloring  a tuo  node  coaplete  graph  uith  one  color  can  be  represented  as  in 


Figure  2.1 


In  tha  figure,  (red  green) j and  (rad  green) 2 are  the  initial  doaains  of 
values  for  Xj  and  X2  respectively,  the  predicate  at  each  node  is  "colored 
red"  and  the  binary  predicate  betueen  the  nodes  ie  "is  not  the  ease  color 


flackuorth  distinguishes  three  levels  of  inconsistency  for  a constraint 


netuork,  uhich  represent  conbinatlons  of  values  uhich  cannot  participate  in 


any  solution  to  the  global  constraint.  The  first  and  Most  obvious  is  node 


inconsistency.  Here  the  potential  donain  of  values  for  X,  and  X2  is  given 


as  red  and  green,  but  the  unary  predicate!  specify  red.  He  can  iaaed lately 


• lininate  green  from  both  nodes,  as  in  Figure  2.2 


Figure  2.2 
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The  next  level  of  Inconeietoncy  it  arc  inconel etency.  the  arc  from  Xj 
to  X2  ie  incone  latent  becauee  for  a value  in  Xj,  namely  "red",  there  doee 
not  exiet  any  value  ag  in  X2  euch  that  red  and  ^ together  eatiefy  the 
relation  "red  ie  not  the  eame  color  ae  ag".  To  remedy  thie  Inconai etency 
Me  reeove  red  from  Xj,  and  eiailarly,  from  X2.  Thie  cute  doun  our  eearch 
apace  all  right:  unfortunately,  in  thie  caee  it  reflecte  the  fact  that  the 
problem  ie  impoeeible.  There  ie  no  global  eolution,  i.e.  the  netuork  ie 
uhat  I call  "unaatiaf iable". 

It  ie  entirely  poeeible  for  a netuork  to  have  no  arc  incone ietenciee, 
and  atill  be  unaatiaf  iable.  Conaider  the  problem  of  coloring  a complete 
three  node  graph  uith  tuo  colore,  repreeented  in  Figure  2.3. 


Figure  2.3 

Aeouae  the  eet  of  poeeible  valuee  for  each  variable  ie  (red  green)  and  the 
binary  predicate  betueen  each  pair  again  apecifiee  "ie  not  the  eame  color 
ae". 

Thie  netuork  ie  arc  coneiatent,  e.g.  given  a value  "red"  for  Xj,  mo 
can  chooee  "green"  for  X2I  red  I a not  the  eame  color  ae  green.  Yet 
obvlouely  there  ie  no  uay  of  chooeing  eingle  valuee  a^,  a2,  *3*  for  Ki>  *2' 
and  Xg,  ouch  that  all  three  binary  conatrainte  ara  eatlefiad 

1 i 

elmul taneouely.  If  ue  chooee  red,  for  Xj,  for  example,  ue  are  forced  to 
chooee  green  for  X2  to  eatiefy  the  conetraint  betueen  X,  and  X2«  Thie 
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force*  a choice  of  red  for  Xj,  which  forces  a choice  of  green  for  Xj, 
already  picked  to  be  red. 

Nevertheless,  It  nay  be  helpful  to  remove  arc  inconsistencies  from  a 
network.  This  involves  comparing  nodes  uith  their  neighbors  as  we  did 
above.  Each  node  must  be  so  comparsdt  however,  comparisons  can  cause 
changes  (deletions)  in  the  network  and  so  the  comparisons  oust  be  Iterated 
until  a stable  network  is  reached.  These  iterations  can  propagate 
constraints  some  distance  through  the  network.  The  comparisons  at  each 
node  can  theoretically  be  performed  in  parallel  and  this  parallel  pass 
i terated. 

Thus  removing  arc  inconsistencies  involves  several  distinct  ideast 
local  constraints  are  globally  propagated  through  iteration  of  parallel 
local  operations.  It  remains  to  be  seen  which  aspects  of  this  process  are 
most  significant  to  its  application.  Ths  parallel  possibilities  may  prove 
to  be  particularly  important!  however,  at  ths  moment  serial 
implementations  are  used  in  practice. 

Ualtz  "filtering”  algorithm  for  scene  labelling  (23)  is  the  paradigm 
example  of  an  arc  consistency  algorithm.  Ualtz  wishes  to  attach  labels  to 
the  lines  in  a line  drawing  indicating  their  semantic  interpretations  as 
convex,  concave  or  occluding  three-dimensional  edges.  The  line  drawing 
itself  functions  a*  the  constraint  network.  Vertices  function  as  network 
nodes.  An  individual  vertex  value  consists  of  a label  for  each  of  the 
lines  incident  to  the  vsrtexi  the  set  of  possible  values  is  initially 
conetrainad  according  to  realizable  three  dimensional  interpretations  for 
the  various  types  of  vertices.  The  lines  are  the  arcs  of  the  network  and 
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•ach  rapraaanta  the  ralation  "the  labellings  of  tha  adjacant  vartlcaa  must 
agree  along  this  line". 

Ualtz  filtering  algoritha  (expecialiy  when  further  conatrainad  by 
specifying  initial  labele  for  edge*  on  tha  background)  generally  rasulte  in 
an  aaazlng  combinatorial  reduction!  thousand*  of  poseibl 1 1 1 ties  are  often 
reduced  to  a state  uhara  all  nodes  have  a single  value  raaaining,  thus 
totally  solving  the  problsa  of  obtaining  the  global  solution.  Of  course 
tha  algoritha  does  not  aluays  terainate  with  a unique  value  at  each  node. 
Generally,  in  this  case,  aost  nodes  will  still  have  a unique  value,  uhile  a 
few  nodes  will  have  a seal  I set  of  values  remaining.  Normally  this  final 
state  indicates  that  several  aabiguous  interpretations  are  possible! 
alternative  sets  of  values  that  siaul taneously  satisfy  all  constraints  can 
bo  quickly  found  with  tree  search. 

It  is  perhaps  not  as  well  appreciated  that  this  final  state  eay  also 
be  reached  for  a figure  which  in  fact  admits  no  consistent  labelling.  Thie 
is  to  be  suspected,  however,  given  that  the  filtering  algorithm  only 
achieves  arc  consistency.  Given  the  basic  Huffman  label  sat  (10)  (4)  (not 
Ualtz*  expanded  label  set)  and  applying  the  filtering  algorithm  (without 

first  constraining  the  outside  lines  to  be  occlusions),  the  line  drawing  in 

* 

Figure  2.4  le  left  labelled  as  shown. 
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Figure  2.4 

However,  there  la  no  cone  latent  choico  of  I aba  Is  for  tha  vartlcaa  of  the 
inner  triangle.  In  other  uorde  the  filtering  algor i the  alone  Mill  not 
deternine  if  a line  drawing  la  what  Huff  ear  cal  la  an  "iepoaaible  figure". 

flontaneri  (IS)  hat  developed  a More  powerful  notion  of  inconeletency 
which  Hackworth  call*  path  I ncona latency.  A network  la  path  inconeietent 
* if  there  era  two  nodee  Xj  and  X2  auch  that  a eatiefiee  Xj,  b eatiefiee  X2. 
a and  b together  aatiafg  the  binary  conatraint  between  thee,  yet  there  ia 
•owe  other  path  through  the  network  froa  Xj  to  X2.  auch  that  there  ia  no 
aat  of  value  a,  one  for  each  node  along  the  path,  which  include  a a and  b, 
and  can  eiaul taneoualy  aatiafy  all  conetrainte  along  the  path.  For 
exaaple,  the  network  In  Figure  2.3  ia  path  Inconalatentt  red  eatiefiee  X}. 
green  Xg,  red  ia  not  the  eaee  color  aa  greeni  however,  there  I a no  value 
for  X2  which  will  aatiafy  the  conetrainte  between  Xj  and  X2,  and  between  X2 
and  Xg,  while  Xj  la  red,  X3  la  green. 

ftontanarl  givea  an  algorltha  that  eaaentially  raeovea  path 
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inconsistencies  fro*  a network.  However,  path  consistency  does  not 
necessarily  inaura  aatiaf  iabi  I ity  aithar,  aa  powerful  aa  it  aounda. 
Conaidar  tha  probiaa  of  coloring  tha  complete  four  noda  graph  with  three 
co I ora  (Figure  2.5). 


Figure  2.5 


Each  node  containa  red,  green  and  blue,  and  each  arc  again  repreaenta  the 
relation  "ia  not  tha  ease  color  aa".  In  particular,  _th  conaietency  doaa 
not  fully  determine  the  aet  of  values  aatiafying  tha  global  conetraint, 
which  in  thie  incone  latent  caaa  ia  tha  empty  aat. 

In  aueeary,  arc  and  path  conaietency  algor i thee  may  reduce  the  aearch 
apace,  but  do  not  in  general  fully  synthesize  tha  global  conetraint.  Uhen 
there  are  multiple  eolutlone,  additional  aearch  uill  be  required  to  apecify 
the  aeveral  acceptable  combinations  of  values.  Evan  a unique  solution  may 
require  further  search  to  determine,  and  the  consistency  algorithms  nay 
even  fail  to  reveal  that  no  solutions  at  all  exist. 
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3.  An  extended  theory 

As  the  coloring  problem  suggests,  the  general  problem  of  eynthesizing 
the  global  conetraint  la  NP-completa  (5),  and  thua  unlikely  to  have  an 
efficient  (polynomial  time)  aolution.  On  the  other  hand  the  experimental 
reaulte  of  Ualtz,  and  the  theoretical  atudiea  of  Hontanari,  suggest  that  in 
apecific  applications  it  may  be  possible  to  greatly  facilitate  the  search 
for  solutions.  I uill  present  an  algorithm  for  synthesizing  the  n-ary 
constraint  defined  by  a set  of  constraints  on  subsets  of  n variables.  It 
may  be  of  substantial  benefit  in  applicatione  where  pruning  of  arc  and  path 
inconsistencies  still  leaves  many  possibilities  to  be  searched. 

There  are  two  key  observations  that  motivated  the  algorithm.  . 

1.  Node,  arc  and  path  coneistency  in  a constraint  network  for  n 
variables  can  be  generalized  to  a concept  of  k-consistency  for  any  ksn, 
where  n-consistency  constitutes  a natural  notion  of  global  consistency. 

2.  The  given  conetrainte  can  be  represented  by  nodee,  as  opposed  to 
links,  in  a constraint  network!  us  can  add  nodes  representing  k-ary 
constraints  to  a constraint  network  for  all  ksn  (whether  or  not  a 
corresponding  k-ary  constraint  is  given) i and  we  can  then  propagate  these 
constraints  in  this  augmented  net  to  obtain  higher  levels  of  consistency. 

By  successively  adding  higher  level  nodes  to  the  network  and 
propagating  constraints  In  the  augmented  net,  we  can  achieve  k-ary 
consistency  for  all  k.  Ue  do  not  need  to  restrict  the  given  constraints  to 
binary  relations.  Ruling  out  lower  order  inconsistencies  in  stages 
progressively  reins  in  the  combinatorial  explosion.  The  final  result  is  a 
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globally  consistent  network,  where  the  n-ary  node  specifies  explicitly  the 
n-ary  constraint  we  seek  to  synthesize.  No  further  search  ie  required. 
The  reet  of  this  paper  will  present  the  algorltha,  along  with  a eufflcient 
theoretical  base  to  justify  Its  operation. 


4.  A prellalnary  axaeple  of  the  synthesis  algorltha 


I Mill  give  a crude  exaaple  of  the  synthesis  algorltha  In  operation, 
by  way  of  aotivation  for  the  foraal  description  which  follows.  The 
presentation  in  this  section  is  intentionally  sketchy. 

Suppose  we  are  given  the  following  constraints  on  variables  Xj,  X2. 
X3«  The  unary  constraint  Cj  specifies  that  Xj  aust  be  either  a or  b,  i.e. 

( Cj«(a  bl . Similarly  C2-  (e  fl  and  Cj»(c  d gl.  The  binary  constraint  on  Xj 

and  X3  specifies  that  either  Xj  is  b and  X2  is  e,  or  X^  is  b and  X3  Is  fi 
C12-(be  bfl.  Likewise  C^-lbc  bd  bg)  and  C23-  led  fgl . 

Ue  wish  to  dsteraine  what  choices  for  Xj,  X2.  X3,  if  any,  can 
aiaul taneously  satisfy  all  these  constraints.  Ue  begin  building  the 
constraint  network  with  three  nodes  representing  the  unary  constraints  on 
the  three  variables,  as  shown  in  Figure  4.1. 

Is  fl2 

(a  bl  j Ic  d glj 

l , 

Figure  4.1 
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N**t  ue  add  nodea  representing  the  binary  conatrainta,  and  link  thaa  to  the 
unary  conatrainta  ae  shoun  In  Figure  4.2  (a.g.  (be  bfl12  represents  C12>. 


lba_  bf I j2  lad  fgl 23 
(a  bl  j — (be  bd  bgl13— Ic  d g»3 

Figure  4.2 

After  ue  add  and  link  node  Cj2  ue  look  at  node  Cj  and  find  that 
alaaant  a doaa  not  occur  in  any  aaaber  of  C12*  Ua  delate  a from  Cj. 
Siailarly,  ue  delete  c froa  C3  after  adding  C23.  Tha  conatraint  natuork  now 
appear  a aa  in  Figure  4.3. 


la  fl, 

(be  bflj2  (ad  ^ Ol  23 
Ul  blj — (be  bd  bglj3 — Id  d gl3 


Figure  4.3 


Nou  froa  C3  ua  look  at  Cj3  and  find  that  there  la  an  alaaant  be  in  Cj3 
which  requires  c aa  a value  for  X3,  uhile  c la  no  longer  in  C3.  Ua  raaove 
be  froa  Cj3#  aa  in  Figure  4.4. 
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<«d^f  o>  23 


j — VU  bd  bgljg—  id  gl  3 


Flgura  4.4 

So  far  ho  hava  aaralg  achieved  a sort  of  "arc  consistency"  (though  mo 
Indicata  tha  restriction  of  tha  pair  be,  aa  Mali  aa  tha  alaaenta  a and  c). 

Naxt,  ua  add  a noda  for  tha  tarnaru  constraint.  No  ordar  three 
conatraint  Mae  given  originally,  00  ua  could  asouae  Initially  the 
"non-conetraint",  all  poaaibla  triples.  However , us  will  take  advantage  of 
the  restrictions  available  froa  the  binary  and  unary  predicates  to 
construct  a aoro  liaited  sat  of  poeeibi I itiaa.  Cj  and  C23  together  allow 
only  tha  following  set  of  triples:  (bad  bfg).  Ua  use  this  aa  tha  ternary 

noda  and  link  it  ta  tha  binary  nodes  aa  shown  in  Figure  4.5. 


is  f)> 


(bl 


(be  bfijj tbed^bfg)  £23 — (ad  fgljrj 


- (bd  bgi  J3  - 
Figure  4.5 


Id  g>3 


Ua  look  at  tha  new  noda  froa  its  neighbors  and  vice  versa,  aa  ua  did 
earlier,  to  insure  consistency  of  the  aort  wa  obtained  earlier  between 
neighboring  nodes.  Cj3  ia  consistent  with  the  new  noda:  bd  is  part  of 
bad,  bg  part  of  bfg,  Sialiarly  and  C23  are  consistent  with  the  ternary 


nod*.  If  n*c*t**rg,  u*  could  propagate  delation#  around  until  local 
conaiatancg  la  achieved  on  thie  augaented  network.  However,  in  thie  caee, 
the  network  ia  already  stable)  no  further  changes  are  required. 

The  ternarg  node  represents  the  synthesis  of  the  given  constrainte. 
There  are  two  wags  to  siaul taneouslg  satisfg  the  given  constraints!  Xj-b, 
XjM,  Xj«d  or  Xjab,  )<2“f,  X-j*g. 


5.  Basic  definitions!  constraint  expressions,  constraint  networks  and 


eatief iabi I i tg 


This  section  presents  several  definitions  needed  to  state  the  problea 
and  Its  solution  preciselg. 

Ue  are  given  a set  of  variables  Xj Xn  which  aag  take  on  values 

froa  a set  of  universes  Uj U„  respectively.  Ue  will  aesuae  the  U,  to 

be  discrete,  finite  doaaina.  Let  I-fl  2 ...  nl.  Hang  of  our  definitions 
will  be  aad*.  for  ang  subset  Jcl.  Ue  denote  bg  Xj  the  indexed  set  of 
variables  IXjlj(j.  A value  a,  in  U,  will  be  called  an  instantiation  of 
X|.  An  instantiation  of  a set  of  variables  Xj,  denoted  bg  eJ(  'is  sn 


indexed  set 


Huee  Idjljjj. 


A conetraint  on  Xj,  denoted  Cj.  I*  a eat  of  instantiations  of  Xj. 
The  "Indexed  set"  notation  lapliee  that  there  la  a function,  a,  fro*  J 
onto  the  instantiation  aj,  which  serves  to  indicate  which  aeeber  of  *j 
instantiates  which  variable!  the  value  of  a at  j,  denoted  *j,  is  the 
Instantiation  of  Xj.  U*  could  also  represent  *j  aa  an  ordered  set  or 


. 
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■-tuple,  where  ■ it  the  number  of  elements  in  the  set  J (called  the 

cardinal  ity  of  J and  denoted  |J|)i  aj>(aj^,...,aj  ),  In  Uj(,  j j < jK 

for  I <K,  I, k«l Thu*  Cj  eeg  be  thought  of  a*  an  e-arg  relation.  I 

have  found  it  uaeful,  however,  to  uee  eet  notation  rather  than  refer  to 

croae  product*  or  predicatee  in  the  preeentation  which  followe.  Given  aj, 

"ajtaj"  will  denote  the  inetantietion  of  Xj  contained  in  aj. 

A conetraint  expression  of  order  n ie  a conjunction  of  conetrainte 

C»  A Cj,  one  constraint  for  each  eubset  J of  I (except  the  eaptg  eubeet). 
J«2l 

Noraallg  we  will  not  be  axplicltlg  given  constraints  for  all  Jcl t 
however,  we  can  assuee  theg  exist,  with  no  loss  of  general itg,  as  the  "non 
constraint"  for  Xj  can  alwsge  be  specified,  the  eet  of  all  combinations  of 
eleaents  from  the  domains  of  the  variables  in  Xj. 

Ue  sag  that  an  Instantiation  aj  satisfies  a constraint  Cj  if  aj<Cj. 
The  instantiation  aj  sstisflee  a conetraint  C^,  HcJ,  if  the  eet 
(SjCejl  j(^,  which  ue  call  aj  restricted  to  H,  is  a member  of  C^.  An 
instantiation  aj,  where  |J|»k,  K-satief lea  a constraint  expression  of 
order  nfck  if  aj  satisfies  the  conetrainte  for  all  HcJ.  If  an 
Instantiation  aj  n-setiefiee  the  conetraint  expression  of  order  n,  we  sag 
that  a|  eat i ef las  the  expression.  A constraint  expression  C ie 
h-setlsf  iable  if  for  all  cardinal  itg  k subsets  J of  I,  there  exists  an  aj 
such  that  aj  k-setisfies  C.  If  C of  order  n ie  n-satief iable  It  Ie  said  to 
be  eatisf iable. 

A conjunction  of  constraints,  a conetraint  expression,  defines  another 

constrainti  the  set  of  all  instantiations  aj  which  satisfy  the  conetraint 

* 

expression.  Our  central  problem  is  to  egnthesize  the  order  n conetraint  on 

O 
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Xj  defined  bg  the  constraint  expression,  i.e.  to  detereine  explicitlg  the 
■at  of  Inetantiatlons  t|  which  sieul taneouelg  aatiefg  all  tha  given 
conetrainte.  An  instantiation  a(  which  satisfias  C is  called  a solution  of 
the  constraint  expression. 

A constraint  network  of  order  k in  n variables,  ksn,  is  a set  of 
constraints  called  nodes,  Nj,  for  each  Jel,  |J|Sk,  where  a I ink  is  said  to 
axiet  batwean  Nj  and  If  HcJ  and  |H|*|J|-1.  Linked  nodes  are  called 
neighbors.  A constraint  network  of  order  n in  n variables  will  be  called  a 
ful  I conetralnt  network.  A node  Nj  Is  said  to  correspond  to  a given 
constraint  Cj  If  Nj-Cj,  i.e.  each  instantiation  of  one  is  a rneeber  of  the 
other.  A full  constraint  network  in  n variables  corresponds  to  a 
constraint  expression  of  order  n if  each  node  Nj  In  the  network 
corresponds  to  the  constraint  Cj  In  the  expression.  The  order  of  a node 
Nj,  or  a constraint  Cj,  is  the  cardinal  itg  of  J. 

For  exaaple,  the  network  In  Figure  5.1  corresponds  to  the  constraint 

expression  C-  A Cj  , where*  l-ll  21,  Cj-lr  g) , C2-lr  g) , C12-lrg  grl . 

J(2l 

<1  avoid  oat  notation  in  tha  subscripts  for  siaplicitg.) 

(p  glj Irg  gr)12 Ir  gl2 

Figure  5.1 

Thle  is  obviouelg  a representation  of  tha  problaa  of  coloring  a two  node 
graph  with  two  colors. 

As  nodes  are  constraints  wa  are  able  to  restate  all  tha  above 
definitions  Involving  satlsf  lebi  I Itg  in  taras  of  nodes  and  networks,  rathar 
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than  constraint*  and  constraint  expressions.  In  particular  us  can  speak  of 
an  instantiation  aj  satisfying  a nods  ^ for  HcJ.  Us  also  Mill  uant  to 
talk  about  aj  satisfying  for  HaJ.  Us  Mill  say  that  aj  satisfies  N^, 
HsJ,  If  there  SKlsts  an  a^  In  such  that  laj(aHl  jfj-aj,  l.a.  there  is  an 
instantiation  Mhlch  satisfies  ^ Mhos*  restriction  to  J is  aj. 


6.  Constraint  propagation 

t 

Ue  can  nou  define  the  basic  constraint  propagation  aechanisa.  To 
Jocal  lu  propagate  the  constraint  Nj  to  a neighboring  constraint  N^,  reaove 
froa  all  a^  which  do  rot  satisfy  Nj.  Global  propagation  is  defined 
recursively.  To  global  lu  propagate  a constraint  Nj  through  a neighboring 
constraint  N^t  first  locally  propagate  Nj  to  then,  if  anything  uae 
reaoved  froa  by  the  local  propagation,  globally  propagate  through  all 
its  neighbors  except  Nj.  To  propagate  a constraint  Nj,  globally  propagate 
Nj  through  all  its  neighbors.  The  propagation  procedure  is  eiailar  to  an 
arc  consistency  algoritha.  hack  mot  th  discusses  efficient  serial  algor  ithas 
for  arc  consistency  (11).  Of  course,  a parallel  iapleaentat ion  is 
possible. 

A constraint  network  Is  said  to  be  relaxed  if  ue  can  propagate  every 
constraint  Nj  in  the  network  without  causing  any  change  (deletions  froa 
nodes)  in  the  net.  The  relaxation  of  a constraint  network  is  the  network 
obtained  by  propagating  all  nodes  of  the  network.  (The  propagation 
obviously  teralnates  in  a relaxed  network.) 


18 


7.  Sun thati ■ algorithm 

Urn  arm  now  ready  to  state  the  synthesis  algorithm.  The  claim,  to  be 
proven  in  section  18,  is  that  this  algorithm,  given  a constraint 
expression,  produces  a constraint  network  whose  order  n node  corresponds  to 
the  order  n constraint  defined  by  the  constraint  expression. 

ALGORITHM! 

Given  C-  A Cj.  Ue  define  the  algorithm  inductively! 

JC21 

STEP  li  Construct  a constraint  network  with  nodes  Nj  corresponding  to 
constraints  Cj  in  the  given  constraint  expression,  for  all  Jd  of 
cardinal i ty  one. 

STEP  k+li  For  all  Jcl  of  cardinality  k+li 

Add  the  node  Nj  to  the  network  corresponding  to  the  given  constraint 

Cj.  Link  Nj  to  al  I such  that  H is  a cardinality  k subset  of  J. 

0 

Locally  propagate  to  Nj  from  each  of  its  neighbors.  Propagate  Nj. 

For  a constraint  expression  of  order  n,  the  algorithm  is  run  for  n 
ateps.  The  result  Is  a full  constraint  network,  where  N|  corresponds  to  C. 

The  next  section  will  present  several  examples  of  the  algorithm  in 
operation.  First  a few  general  observations.  The  network  produced  by  this 
algorithm  is  the  relaxation  of  the  network  corresponding  to  C.  He  could 
have  obtained  It  simply  by  building  the  corresponding  order  n network  and 
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propagating  each  node.  Bg  proceeding  in  etagee  Me  take  advantage  of  the 
eliaination  of  poseibi  I i ties  that  eag  occur  at  each  etage  to  Mitigate 
combinatorial  explosion.  Ue  take  this  principle  further  and  propagate  each 
node  as  it  is  added,  before  adding  another.  A good  heuristic  uould  be  to 
add  earlier  those  nodes  uhich  exert  a heavg  constraint,  o.g.  where  Cj  ie 
small.  The  propagation  of  these  constraints  mag  eliminate  elements  from 
nodes  used  in  constructing  later  constraints.  If  Cj  is  the  non-constraint 
ue  can  construct  Nj  initial  Ig  from  some  and  Nj_^,  where  H is  a 
cardinal  itg  k subset  of  J,  preferablg  the  one  for  uhich  |Nj|x|Nj_^|  ie  a 
minimum.  (Add  to  each  member  of  each  member  of  Nj_^. ) 

Other  refinements  are  clear  Ig  possible.  Provision  should  be  made  for 
early  termination,  e.g.  as  soon  as  one  node  becomes  emptg.  Propagation  can 
be  simplified,  a.g.  by  noting  non-constraints,  or  using  complements  of 
nodes.  Additional  links  could  permit  direct  propagation  between  a node  Nj 
and  the  nodes  for  all  subsets  of  J. 

It  is  generally  redundant  to  require  all  non-constraint  nodesi 
basically  we  only  need  one  "path"  up  to  the  n-ary  node  for  every  "real" 
constraint.  Consider  a constraint  expression  on  four  variables  where  only 
the  binary  constraints  are  really  specified  (the  others  are 
non-constraints).  Only  the  binary  constraints  can  raally  have  any  affect 
on  the  global  solution.  Three  ternary  nodes  are  sufficient  for  the  network 
constructed  by  the  algorithm.  If  the  fourth  ternary  node  rules  out  any 
element  of  the  order  four  node,  it  is  only  reflecting  a binary  constraint, 
which  la  reflected  In  one  of  the  other  ternary  nodes.  On  the  other  hand  we 
may  be  Interested  In  the  effects  on  non-cone tr a i ate  of  the  pr,opagation 
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process.  In  general  the  pruning  procast  of  the  algorithm  progressively 
makes  explicit  at  Nj  restrictions  on  instantiations  of  Xj  that  are  not 
originally  given  by  Cj,  but  rather  implied  by  the  other  constraints.  In 

the  final  netuork  produced  by  the  algorithm  every  member  of  ever.y  Nj  ie 

«> 

part  of  some  solution  of  the  constraint  expression.  (In  particular,  ue 
have  derived  the  "minimal”  netuork,  Hontanarl’s  "central  problem"  (151.) 


8.  Further  examples!  graph  coloring,  scene  labelling,  graph  isomorphism 

As  the  synthesis  problem  is  such  a general  one,  the  synthesis 
algorithm  has  many  potential  applications.  Graph  problems,  of  course,  lend 
themselves  particularly  to  a constraint  netuork  formulation.  I present  in 
this  section  three  applications  uhich  ul 1 1 serve  to  illustrate  the 
algorithm,  and  are  of  some  independent  interest  as  uell. 

As  ue  uould  expect  from  the  discussion  in  section  2,  the  graph 
coloring  problem  can  easily  be  represented  as  a constraint  netuork.  Given 
a graph  G,  and  a set  of  colors,  ue  construct  a constraint  netuork  from  G as 
fol  I oust  Each  node  of  G is  replaced  by  the  unary  constraint  representing 
the  set  of  colors.  If  there  Is  an  edge  betueen  nodes  in  G,  ue  replace  it 
by  a binary  constraint  linked  to  the  nodes  uhich  represents  "is  not  the 
same  color  as*.  If  there  is  no  link  betueen  nodes  In  G,  ue  add  the 
non-constraint  betueen  the  nodes. 

Let  us  consider  tuo  examples.  First  consider  the  problem  of  coloring 
a coaplate  three  node  graph  uith  three  colors.  Figures  8.1a,  8.1b  and  8.1c 
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•hoM  the  constraint  network  after  stop*  ona,  two  and  three  of  the 
algor  I the,  uhara  tha  nodes  Nj,  Nj,  N3  are  all  the  sot  (r  g bl , Nl2,  Nl3  and 
N23  all  equal  (rg  rb  gr  gb  Or  pg)  and  N123-lrgb  rbg  brg  bgr  grb  gbrl . tha 
ala  possible  colorings. 


Figure  8.1 

Ue  could  construct  a natuork  of  the  sort  ue  used  In  section  2,  for  this 
problea.  However  the  network  would  be  path  consistent!  arc  and  path 
consistency  algor  I thus  would  not  rule  out  any  aleaents  at  the  nodoe. 

Consider  now  tha  problea  of  coloring  a eoaplete  four  node  graph  with 
three  colors,  which  wa  used  In  section  2 to  illuetrate  that  path 
cone  latency  is  not  a sufficient  condition  for  satisfiability.  After  tha 
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third  step  of  tho  algorithm  uo  Mould  hava  four  ternary  nodes,  aach  equal  to 
tha  ternary  node  in  the  previous  example. 

At  tha  beginning  of  tha  fourth  etap  ue  uee  N^3  and  to  construct  an 
order  four  nodet  ^234  ■ Irgbr  rgbg  rgbb  rbgr  rbgg  rbgb  brgr  brgg  brgb  bgrr 
bgrg  bgrb  grbr  grbg  grbb  gbrr  gbrg  gbrbl . Local  propagation  froa  the  other 
ternary  nodee  quickly  reduces  the  order  four  node  to  the  eepty  set  (and 
thia  constraint  propagates  back  doun  to  reeove  all  aleeenta  from  all  the 
nodes).  No  instantiation  of  the  order  four  node  Mill  siaul taneouely 
satisfy  the  four  ternary  nodes.  Unsatisf iabi 1 1 ty  is  demonstrated. 

These  examples  are  rather  perverse  cases,  of  course,  though  they  do 
illustrate  points  uith  respect  to  the  discussion  in  section  2.  Applications 
in  the  scene  labelling  domain  generally  involva  more  propagation  than 
occurs  in  the  coloring  problem.  The  synthesis  algorithm  does  function  as  a 
test  for  impossible  figures.  It  also  finds  all  the  Interpretations  in  an 
ambiguous  figurs.  You  may  uant  to  simulate  the  algorithm  on  a simple 
figure  like  that  in  figure  8.2. 


After  the  Halts  filtering  algorithm  is  run  on  the  Huffman  label  set 
<ui  thou t additional  constraints  on  the  background  label  I Inge)  there  are 
three  labels  loft  at  aach  order  tuo  vertex  and  tuo  at  each  order  three 


vertex 
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For  a final  exaaplo,  uo  taka  graph  iaoaorphiaa.  Gi van  two  graphs  G 
and  H,  which  wa  wish  to  taat  for  iaoaorphiaa,  conatruct  a conatraint 
network  froa  G aa  follows.  (If  H has  oora  nodas  than  G tha  algoritha  will 
aaak  iaoaorphic  aappinga  of  G onto  subgraphs  of  H.)  Raplaco  each  node  of  G 
with  a unary  conatraint  node  containing  all  tha  nodes  of  H.  (If  wa  allow 
loopa,  edges  froa  a node  to  itself,  tha  unary  constraint  on  a node  in  G 
ulth  a loop  will  bo  "has  a loop  in  H",  on  a node  in  G without  a loop,  "has 
no  loop  In  H".  Uo  could  also  incorporate  additional  unary  conatralnta  auch 
aa  tha  order  of  tha  vortex  (221.)  Replace  each  edge  between  nod  a a a and  b 
in  G uith  a binary  conatraint  node,  linked  to  the  unary  nodee  for  a and  b. 
This  binary  node  will  represent  the  constraint  "these  two  (diatinct)  nodes 
share  an  edge  in  H".  In  other  words  tha  binary  constraint  will  contain  a 
pair  xy  if  and  only  If  there  ia  an  edge  between  x and  y in  H.  Between  two 
nodae  which  do  not  share  an  edge  In  G we  also  place  a binary  node,  linked 
to  thee,  but  representing  the  conatraint  "these  two  (distinct)  nodee  do  not 
ahara  an  edge  in  H". 

For  exaaplet  given  tha  graphs  G and  H In  Figure  8.3 

1 

, c ^ , 3 

2 

G H 

Figure  8.3 


we  produce  the  conatraint  network  in  Figure  8.4. 
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Propagating  constraint*,  us  obtain  th*  nstuork  in  Flgurs  8.6. 
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Figure  8.5 
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Now  adding  th*  ternary  nod*  u*  obtain  Figure  8.6. 


<1  21, 
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Figure  8.6 
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This  network  is  relaxed.  Ths  ternary  nods  reprsssnts  the  two  possible 
isomorphisms:  s->l,  b->2,  c->3  and  a->2,  b->l,  c->3.  (The  algorithm  also 

finds  isomorphic  subgraphs  along  ths  uag. ) 

In  the  above  applications,  ths  desired  global  stats  is  defined  in 
terms  of  local  constraints.  Often  us  first  face  an  analysis  problem: 
choosing,  or  learning,  a set  of  local  constraints  that  specify  or 
approximate  the  desired  global  stats  (201.  (An  important  concern  will  be 
the  choice  of  a "good"  constraint  expression,  i.e.  one  that  can  bs 
synthesized  efficiently.)  As  us  explore  various  applications,  it  Mill,  of 
course,  be  equally  important  to  develop  theoretical  methods  for  analyzing 
the  performance  of  the  algorithm  in  a given  domain. 

o . ■ - . | 

9.  Consistency  and  comp  lateness 

The  synthesis  algorithm  operates  by  removing  higher  and  higher  level 
inconsistencies  until  a global  consistency  has  been  achieved.  In  this 
section,  I define  this  sequence  of  consistency  states,  and  also  define  a 
concept  of  completeness  which  us  ulll  uant  to  apply  to  a network. 

A node  Nj  of  order  k Is  ^-consistent  with  a constraint  expression  C 
If  all  members  of  Nj  k-eatlefy  C.  A network  of  order  k or  greater  le 
k-conel stent  with  C If  all  nodes  of  order  k are  k>consl stent  with  C.  If  a 

full  constraint  network  of  order  n Is  n-conalstent  with  a constraint 

% 

expreaaion  C of  order  n wo  say  that  it  is  consistent  with  C. 

; 
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A nod*  Nj  of  order  k is  k-complpte  for  C if  any  instantiation  aj 
which  k-satisfles  C is  a member  of  Nj.  A network  N is  k-complete  for  C if 
every  node  of  order  k is  k-coeplete.  An  n-complete  full  constraint  network 
of  order  n is  said  to  be  complete. 

A few  comments  may  be  In  order  to  relate  the  consistency  notions 
described  in  this  section  to  the  background  discussion  in  section  2.  For 
networks  of  unary  and  binary  constraints,  k-consistsncy  implies  that  if  we 
pick  values  of  any  k-1  variables  from  the  unary  nodes,  and  a kth  variable, 
there  will  be  a value  of  the  kth  variable,  at  the  unary  node,  such  that  the 
k values  together  satisfy  all  predicates  involving  ths  k variables  (i.e. 
they  form  an  instantiation  of  Nj  where  J is  the  set  of  k variables 
chosen).  This  indicates  that  l-consistency  of  a constraint  network  implies 
node  consistency  of  the  corresponding  network  of  the  type  described  in 
section  2,  2-consistency  implies  arc  consistency  and  3-cons i s tency  implies 
path  consistency.  The  first  two  are  obviouss  the  latter  requires  reducing 
path  consistency  to  the  three  node  case,  which  is  done  by  induction  in  a 
theorem  of  Montanari. 

Suppose  we  seek  a global  solution  by  using  depth  first  tree  search  on 
the  elements  remaining  in  the  unary  nodes  of  a k-consi stent  network  of 
unary  and  binary  constraints.  Backup  will  only  be  initiated  below  the  kth 
level.  If  the  network  is  consistent,  there  will  be  no  backup.  Even 
better,  and  for  arbitrary  relaxed  constraint  networks,  we  can  choose  an 
order  k node,  and  use  its  members  as  the  alternative  paths  through  the 
first  k lavele  of  a search  tree,  only  really  doing  tree  search  on  the 
remaining  n-k  nodes.  Of  course,  If  we  have  achieved  full  coneietency  the 
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aaebara  of  the  order  n nodt  aro  th»  solutions  and  no  further  March  la 
required. 


10.  Synthesis  theoroe 

Ue  are  nou  ready  to  state  the  theoree  which  justifies  the  eynthesie 
algor  I the. 


THEOREM!  The  relaxation  of  the  network  corresponding  to  a constraint 

expression  C*  A Cj  Is  consistent  and  coeplete  with  respect  to  C. 

JC21 

The  proof  will  be  by  induction.  Consistency  and  coapleteness  of  order 
one  are  obvious.  Our  Induction  hypothesis  is  that  the  network  ie 
k-coneietent  and  k-coepletei  we  wish  to  prove  k+l-consl etency  and 
k+l-coep I etenees. 

Consletencyi  Ue  want  to  show  that  all  Nj,  for  J any  cardinality  k+1 
subset  of  I.  are  k+1  -consistent.  Nj,  before  relaxation,  corresponded  to 
Cj,  so  included  nothing  which  did  not  satisfy  Cji  relaxation  does  not  add 
any  el  scents  to  a node.  Suppose  there  exists  an  aj  In  Nj  ouch  that  aj 
does  not  satisfy  C^,  for  soee  proper  subset  H of  J,  i.e.  Sj  restricted  to 
H ie  not  e eeeber  of  C^.  Pick  a set  C of  cardinality  k such  that  HcGcJ. 
Because  of  the  local  propagation  during  the  relaxation  process,  ue  know 
that  aj  satisfies  Nq.  Thus  aj  restricted  to  G,  Sq,  is  a eeeber  of  Nq.  As 
the  network  ie  k-consi stent  Sq  restricted  to  H ie  a eeeber  of  C^.  But  ^ 
restricted  to  H ie  aj  restricted  to  Hi  contradiction. 


■* 
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Coapleteneeei  Consider  any  aj  not  in  Nj,  for  J any  cardinality  k+1 
aubaat  of  I.  Thara  ara  tuo  poaalbi  I itlaa.  If  aj  uaa  not  In  Nj  bafora 
ra taxation,  than  aj  doaa  not  aatiaty  Cj.  If  aj  uaa  raaovad  during  tha 
ra taxation  procaaa,  than  aj  doaa  not  aatiafy  NH  for  aoaa  cardinality  k 
aubaat  H of  Ji  by  tha  induction  hypothaaia  aj  raa  trie  tad  to  H doaa  not 
k-aatiafy  C.  In  aithar  caaa,  aj  doaa  not  k+l-aatlafy  C. 

Thara  ara  aavaral  iaaadiata  corollariaa. 

Corollary  It  Nj  corraaponda  to  tha  ordar  n conatraint  dafinad  by  tha 
conatraint  axpraaaion  C. 

Corollary  2t  C ia  aatiaf labia  if  and  only  if  N|  ia  not  tha  aapty  aat. 

Corollary  3t  Tha  conatraint  natuork  conatructad  by  tha  aynthaaia 
algor i tha  operating  on  a conatraint  axpraaaion  C ia  k-conalatant  with  and 
k-coaplata  for  C after  step  k.  Tha  natuork  conatructad  by  tha  algor i tha  ia 
coneietent  ulth  and  coaplata  for  C,  and  Nj  corraaponda  to  C. 
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